Papers with Pareto frontier
From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are often optimized for direct question-answering, but their effectiveness is often undermined by strategically withholding answers. |
| Approach: | They propose an online reinforcement learning-based alignment framework that can quickly adapt LLMs into effective tutors using simulated student-tutor interactions. |
| Outcome: | The proposed model outperforms proprietary models like LearnLM and can be used to enhance interpretability and pedagogical quality. |
From Research to Production and Back: Ludicrously Fast Neural Machine Translation (D19-56)
Copied to clipboard
Young Jin Kim, Marcin Junczys-Dowmunt, Hany Hassan, Alham Fikri Aji, Kenneth Heafield, Roman Grundkiewicz, Nikolay Bogoychev
| Challenge: | Using the dominating submissions to the previous edition of the shared task, we develop improved teacher-student training via multi-agent dual-learning and noisy backward-forward translation for Transformer-based student models. |
| Approach: | They propose to use multi-agent dual-learning and noisy backward-forward translation to improve teacher-student training for Transformer-based student models. |
| Outcome: | The proposed model outperforms submissions to the previous edition of the WNGT efficiency shared task by 4 BLEU points and 10 BLUE points respectively. |
Optimizing Length Compression in Large Reasoning Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Reasoning Models suffer from producing unnecessary and verbose reasoning chains. |
| Approach: | They propose a post-training method that uses a Length Reward and a Compress Reward to remove the invalid portion of the thinking process. |
| Outcome: | The proposed method reduces sequence length by 50% with only a marginal (2%) drop in accuracy. |
ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models (2026.findings-acl)
Copied to clipboard
Tingyun li, Zishang Jiang, Jinyi Han, Xinyi Wang, Sihang Jiang, Han Xia, Zhaoqian Dai, Ma Shuguang, Fei Yu, Jiaqing Liang, Yanghua Xiao
| Challenge: | Existing efficiency-oriented methods attempt to shorten or mix reasoning strategies, yet often degrade reasoning capability. |
| Approach: | They propose a token-level dual-process framework that explicitly decouples efficiency and correctness signals during training. |
| Outcome: | The proposed framework reduces inference cost while maintaining strong reasoning ability across multiple benchmarks. |
LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Modern automatic speech recognition systems rely on encoder-decoder architectures and their encoders are a critical bottleneck for efficient deployment due to high computational intensity. |
| Approach: | They propose a low-rank compression scheme for ASR encoders that leverages the strong low-ranked properties observed in intermediate activations and approximates linear transformations with a chain of low-Rank matrix multiplications. |
| Outcome: | The proposed method reduces inference costs while maintaining transcription accuracy while preserving low-rank properties observed in intermediate activations. |
Towards Higher Pareto Frontier in Multilingual Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing Pareto optimization approaches are limited by the long-tailed distribution of multilingual corpora. |
| Approach: | They propose a Pareto mutual distillation framework that pushes the Paret frontier outwards rather than making trade-offs. |
| Outcome: | The proposed framework pushes the Pareto frontier outwards rather than making trade-offs, the authors show. |
Select-Then-Decompose: From Empirical Analysis to Adaptive Selection Strategy for Task Decomposition in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing task decomposition methods focus on memory, tool usage, and feedback mechanisms, but they often overlook the trade-off between performance and cost. |
| Approach: | They propose a strategy that selects the most suitable decomposition approach based on task characteristics and enhances the reliability of the results through a verification module. |
| Outcome: | The proposed strategy is based on categories of approaches, characteristics of tasks, and configuration of decomposition and execution models. |
MADRA: Multi-Agent Debate for Risk-Aware Embodied Planning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing safety alignment methods, such as RLHF, fall into a Safety-Utility Trade-off, resulting in severe over-rejection of benign household instructions. |
| Approach: | They propose a meta-cognitive Critical Agent that evaluates peer debates using a structured argumentation framework derived from the Toulmin Model. |
| Outcome: | The proposed architecture outperforms existing systems in the SafeAware-VH benchmark. |
Vista-LLM: Decoupled Query-Guided Visual Token Pruning for Efficient Long-Video Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Long-video understanding is bottlenecked by the high cost of processing massive visual tokens. |
| Approach: | They propose a decoupled framework for query-guided visual token pruning . their method reduces visual tokens by 90% and accelerates inference by 98% . |
| Outcome: | The proposed framework reduces visual tokens by 90% and accelerates inference while retaining over 98% of baseline performance on average. |
Improving the Quality Trade-Off for Neural Machine Translation Multi-Domain Adaptation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Building neural machine translation systems to perform well on a specific target domain remains a challenge. |
| Approach: | They propose to train a single NMT system per language pair that performs well across multiple domains. |
| Outcome: | The proposed approach improves the Pareto frontier on this task. |
Balancing Fidelity and Plasticity: Aligning Mixed-Precision Fine-Tuning with Linguistic Hierarchies (2026.findings-acl)
Copied to clipboard
Changhai Zhou, Shiyang Zhang, Yuhua Zhou, Jun Gao, Qian Qiao, Shichao Weng, Weizhong Zhang, Cheng Jin
| Challenge: | Existing quantization-aware fine-tuning methods decouple weight precision and adapter capacity, overlooking that a layer’s ability to adapt is constrained by the information preserved in its frozen weights. |
| Approach: | They propose a framework that jointly optimizes per-layer quantization bit-width and LoRA rank. |
| Outcome: | Experiments on LLaMA and Qwen models show that the proposed framework matches or approaches 16-bit baselines while using substantially less memory. |
Bypassing Neural Evaluations for Fast Audio Editing via Adaptive Trajectory Extrapolation (2026.findings-acl)
Copied to clipboard
Xiaoqian Liu, Zhengkun Ge, Jianjin Wang, Haoran Zhang, Yuan Ge, Kaiyan Chang, Chen Xu, Tong Xiao, Zhengtao Yu, Linfeng Zhang, JingBo Zhu
| Challenge: | Recent advances in audio diffusion models have significantly improved text-to-audio editing via inversion techniques, but these models typically rely on dense, fixed-step sampling trajectories to maintain structural integrity. |
| Approach: | They propose a model-agnostic Adaptive Trajectory Extrapolation framework that accelerates inversion-based editing process by dynamically evaluating only the most critical generative phases. |
| Outcome: | The proposed framework achieves a 3.9 speedup with negligible loss in fidelity. |
TARE: Lightweight Token-Aware Representation Editing for Fine-tuning Transformer-like Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing PEFT methods can be costly and underfit token-level contexts. |
| Approach: | They propose a PEFT method that performs fine-grained, token-specific edits with a small additional inference overhead and minimal tuning. |
| Outcome: | The proposed method outperforms state-of-the-art methods in 8 tasks and GLUE with a minimal tuning overhead and inference overhead. |
Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are inherently dual-use and can be leveraged for both beneficial and harmful purposes. |
| Approach: | They propose a retention-prioritized gradient synthesis framework that decouples task-specific gradient extraction from conflict-aware combination. |
| Outcome: | The proposed method achieves tighter alignment on WMDP Bio and RWKU benchmarks. |
PARIF: Pushing the Pareto Frontier of Instruction Following and Reasoning with Curriculum Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing alignment methods struggle to balance general reasoning with instruction-following (IF) this is hindered by dependency on teacher models, reward hacking, and reasoning-answer inconsistencies. |
| Approach: | They propose a two-stage curriculum learning framework based on Reinforcement Learning from Verifiable Rewards to enhance both IF and general reasoning capabilities. |
| Outcome: | The proposed framework outperforms leading models on six representative IF tasks while achieving a 21.25% relative average improvement over the original model. |
Mixtures of In-Context Learners (2025.acl-long)
Copied to clipboard
| Challenge: | In-context learning is sensitive to the choice of in-con context demonstrations and processing many demonstrations can be computationally demanding. |
| Approach: | They propose a method that uses subsets of demonstrations to train experts via ICL and learns a weighting function to merge their output distributions via gradient-based optimisation. |
| Outcome: | The proposed approach improves on 5 out of 7 classification datasets compared to strong baselines and reduces the inference time needed to achieve the same performance with fewer demonstrations. |
From TDMA to CDMA: A Multi-bit Watermark for Diffusion Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multi-bit watermarking schemes cannot be directly applied to DLMs. |
| Approach: | They propose a multi-bit watermarking framework that encodes the entire watermark message holographically. |
| Outcome: | The proposed framework encodes the entire watermark message across all tokens holographically. |
AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Weight-only quantization is a powerful optimization technique for large language models . pushing below 4 bits often leads to substantial accuracy degradation due to increased quantization error. |
| Approach: | They propose a framework that assigns layer-wise quantization bit-widths to optimize model quality and memory usage. |
| Outcome: | The proposed framework can optimize for large language models under memory constraints. |
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Sparse attention is a promising strategy to extend long-context capabilities in LLMs . but its efficiency–accuracy trade-offs remain unclear due to the lack of comprehensive evaluation . |
| Approach: | They evaluate sparse attention methods across multiple model families and sizes . they find larger sparser models outperform smaller dense ones at equivalent cost . |
| Outcome: | The proposed methods outperform smaller sparse models at equivalent cost and improve the Pareto frontier. |
Faster MoE LLM Inference for Extremely Large Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing inference optimizations for coarse-grained Mixture-of-Experts models implicitly assume a fixed activation budget, which is poorly understood. |
| Approach: | They propose a training-free policy that adapts token-level activation using router confidence and entropy while remaining within the model’s original budget. |
| Outcome: | The proposed skipping policy can provide substantial throughput gains, but optimal static schedules vary significantly across models and routing mechanisms. |
CEBC: Conformal Evidence-Bounded Control for Low-Hallucination Vision–Language Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing mitigation approaches reduce hallucinated object mentions at the cost of degraded generation quality or require expensive retraining and task-specific supervision. |
| Approach: | They propose a lightweight framework for low-hallucination vision–language generation . it uses evidence-bounded minimal editing to revise or suppress unsupported referenced entities . |
| Outcome: | The proposed framework reduces hallucinations while maintaining or improving quality metrics. |
Awakening Dormant Experts:Counterfactual Routing to Mitigate MoE Hallucinations (2026.acl-long)
Copied to clipboard
Wentao Hu, Yanbo Zhai, Xiaohui Hu, Mingkuan Zhao, Shanhong yu, Xue Liu, Kaidong Yu, Shuangyong Song, Xuelong Li
| Challenge: | Sparse Mixture-of-Experts models are vulnerable to hallucinations, authors say . static Top-k routing leaves "specialist experts" under-prioritized for specific tokens . |
| Approach: | They propose a training-free inference framework to awaken dormant experts . they propose 'counterfactual routing' to shift computational resources from syntax-dominant to knowledge-intensive layers . |
| Outcome: | Experiments show that CoR improves factual accuracy by 3.1% without increasing the inference budget. |